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ABSTRACT 

My research is rooted in improving K-12 educational practice 
using motivational facets made possible through adaptive tutoring 
systems. In an attempt to isolate best practices within the science 
of learning, I conduct randomized controlled trials within 
ASSISTments, an online adaptive tutoring system that provides 
assistance and assessment to students around the world. My work 
also incorporates big data analytics through the establishment of 
data driven learning models that promote the use of finite 
assessment to optimize student modeling and enhance user 
motivation. This paper highlights a turning point in my research 
as I transition into PhD candidacy. My contributions thus far and 
my research goals are discussed, with consult sought on how to 
best meld the realms of my work moving forward. An iteration of 
this work has also been published as a Doctoral Consortium at 
AIED 2015 [4]. 
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1. RESEARCH FOCUS 

1.1 Adaptive Tutoring: ASSISTments 

The U.S. Department of Education’s National Educational 
Technology Plan supported the idea that technology will play a 
key role in delivering personalized educational interventions [14]. 
Yet there remains a severe lack of research regarding the 
effectiveness of online learning systems for K-12 education [15]. 
Adaptive tutoring systems offer interactive learning environments 
that allow students to excel while providing teachers a unique 
approach to classroom organization and data-driven lesson plans. 
Before the development of these adaptive platforms, research 
within classrooms was costly and generally required a 
longitudinal approach. As such, much of the evidence that 
supports K-12 educational practice is generalized from studies 
conducted by psychologists in laboratory settings with college 
undergraduates. 

My research acts on this deficit, by conducting controlled trials 
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using student level randomization within ASSISTments, an online 
adaptive tutoring system, to isolate best practices for learning 
outcomes while enriching the user experience. ASSISTments, 
commonly used for both classwork and homework, presents 
students with immediate feedback and a variety of rich tutorial 
strategies. The platform is also a powerful assessment tool, 
providing teachers with a variety of student and class reports that 
pinpoint where students are struggling and enhance classroom 
techniques using real time data. Further, the platform is unique in 
that it allows educational researchers to design and implement 
content-based experiments without extensive understanding of 
computer programming, serving as a shared collaborative tool for 
the advancement of the science of learning [3]. 

1.2 Motivational Trinity 

Essentially, my work seeks to enhance student motivation and 
performance by enriching content through optimized feedback 
delivery, exploring opportunities to make students shareholders in 
the learning process, and attempting to boost motivation and 
proper system usage through improved assessment techniques. 

1.2.1 Feedback Mediums 

Until recently, virtually all feedback within the ASSISTments 
tutoring platform was provided using text, typically with font 
color or typeset signifying important variables. However, adaptive 
tutoring systems offer the opportunity to utilize a variety of 
hypermedia elements, as outlined by Mayer’s multimedia 
principles for the optimal design of e-Learning environments [1]. 
These twelve principles, driven by cognitive theory, promote 
active learning while reducing cognitive load and accounting for 
the average user’s working memory [1]. Educational technologies 
that employ video tend to do so in a manner that resembles 
lectures rather than feedback (i.e., Khan Academy). Thus, the 
introduction of matched content video feedback to the 
ASSISTments platform through brief 15-30 second YouTube 
recordings offered a novel approach to investigating hypermedia 
within an adaptive setting. 

1.2.2 Student Choice 

While platforms like ASSISTments offer a variety of features, few 
make students shareholders in the learning process. Despite the 
fact that users can endlessly customize their experiences with 
commercial products, student preference is not a key element in 
the realm of education. Choice is an intrinsically motivating force 
[11] that has the potential to boost subjective control, or a 
student’s perception of their causal influence over their learning 
outcomes [12]. Feelings of control are balanced by appraisals of 
subjective value, or a student’s perceived importance of her 
learning outcome. By providing the student with choices at the 
start of her assignment, it may be possible to enhance 
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expectancies regarding her performance and thereby enhance 
achievement emotions such as motivation [12]. Considering the 
control-value theory within the realm of an adaptive tutoring 
system for mathematics content may help to explain and 
ameliorate female dropout in STEM fields [2] . Feedback medium 
personalization offers one simple method to examine the 
motivational effect of choice within these platforms. 

1.2.3 Improving Assessment 

Adaptive tutoring systems typically function through measures of 
binary correctness on a student’s first attempt or first action within 
a problem. Within such systems, students who take advantage of 
tutoring feedback are unduly penalized. This creates an 
environment in which students are afraid to use the beneficial 
features of these platforms, or instead, overuse feedback if they 
have already lost credit (i.e., skipping to the answer rather than 
reading a series of hints). The establishment of partial credit 
scoring would help to alleviate these issues, serving to motivate 
student performance while simultaneously offering teachers a 
more robust view of student knowledge. Using data mining 
approaches, partial credit can be defined algorithmically [16] for 
the purpose of enhancing student modeling. Real time 
implementation of these data driven models could offer 
substantial benefits for all parties. 

2. PROPOSED CONTRIBUTIONS 

Thus far, my work has lead to eight peer reviewed articles already 
published or in press, as well as a multitude of projects that are in 
progress. Projects that best highlight my goals as 1 transition to 
my PhD work are described in the following subsections. 

2.1 Published Works 

2.1.1 Video vs. Text Feedback 

The ASSlSTments platform was used to conduct a randomized 
controlled trial featuring matched content video and text feedback 
within the realm of middle school mathematics [7]. Results 
suggested significant effects of video feedback, showing enhanced 
learning outcomes on next question performance after receiving 
adaptive video tutoring, as well as increased efficiency. Further, 
through self-report it was observed that students perceived video 
as a positive addition to their assignment. This study was the first 
of its kind to explore the potential for replacing text feedback, 
already shown to be successful within ASSlSTments [13], with an 
alternate medium. A scaled-up replication of this study is 
currently underway. This work inspired an influx of video content 
into the ASSlSTments platform, providing new opportunities to 
examine the subtleties of video feedback, including a crowd- 
sourced approach to feedback creation. 

2.1.2 Dweckian Motivation 

Moving beyond the use of video feedback and into the realm of 
pedagogical agents, my co-authors and 1 sought to investigate the 
motivational effects of Dweckian inspired mindset training within 
ASSlSTments feedback [10]. A six-condition design was used to 
examine how growth mindset messages promoting the 
malleability of intelligence delivered with domain based feedback 
effected motivation and learning outcomes. Conditions differed on 
elements of audiovisual message delivery, ranging from plain text 
to an animated pedagogical agent. Although limited by a small 
sample size and ceiling effects, analyses across five mathematics 
skills revealed that mindset messages altered student performance 
as measured by persistence, learning gain, and self-reported 
enjoyment of the system (trends, p ~ 0.1). Trends also pinpointed 
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gender differences in response to messages delivered using the 
pedagogical agent. 

2.1.3 Partial Credit Assessment 

By data mining log files from ASSlSTments usage spanning the 
2012-2013 school year, this work established a simple student 
modeling technique for the prediction of next problem correctness 
(time r -I- 1) using algorithmically defined partial credit scores at 
time t [5]. Although traditional modeling approaches and most 
adaptive tutors are driven by binary metrics of student correctness, 
employing partial credit can enhance student motivation and 
promote proper use of system features such as adaptive feedback, 
while allowing teachers a more robust understanding of student 
ability and simultaneously enhancing predictive modeling. 
Predictions gathered using a tabling approach based on maximum 
likelihood probabilities were able to compete with standard 
Knowledge Tracing models in terms of model accuracy, while 
drastically reducing computational costs [5]. 

2.2 Works in Press or in Progress 

2.2.1 Student Choice 

This work served as a pilot study on the addition of student choice 
into the ASSlSTments platform [8]. This line of research 
examines motivation and learning when students are able to invest 
in the learning process. Students were randomly assigned to either 
Choice or No Choice conditions within a problem set on simple 
fraction multiplication. Those given choice were asked to select 
their feedback medium, while those without choice were 
randomly assigned to receive either text or video feedback. 
Results suggested that even if feedback was not ultimately used, 
students who were prompted to choose their feedback medium 
significantly outperformed those who were not. A second iteration 
of this study is currently underway using a new If-Then 
navigation infrastructure that was built because of the significant 
effects observed in the pilot. If previous results are replicated, 
these findings may be groundbreaking in that the addition of 
relatively inconsequential choices to adaptive tutoring systems 
could enhance student motivation and performance. 

2 . 2.2 Content Delivery Patterns 

Motivation and learning outcomes can also be improved by 
making content delivery more adaptive. Recent work within 
ASSlSTments has revealed the benefit of interleaving (or mixing) 
skill content within homework settings [9]. Serving as a 
conceptual replication of previous work in the field, our goal was 
to isolate the interleaving effect within a brief homework 
assignment, as measured by learning gains on a delayed posttest. 
Using a randomized controlled trial, a practice session was 
presented featuring either interleaved or blocked content spanning 
three math skills. This study was unique in that rather than relying 
on a formal posttest, a second homework assignment was used to 
gauge learning gains through average score, hint usage, and 
attempt count. The use of tutoring feedback during posttest 
provided additional dependent variables for analysis while 
allowing students continued learning opportunities. Observations 
revealed that interleaving can be beneficial in adaptive learning 
environments, and appears especially significant for low 
performing students. 

2.2.3 Assessment Enhancing Motivation 

An extension of the work presented in 2.1.3, this research 
examined partial credit scoring using a grid search of 441 
algorithmically defined models through per hint and per attempt 
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penalizations [6]. Binary scoring, as utilized by most adaptive 
tutoring systems, can serve to demotivate students from engaging 
with tutoring feedback and rich system features that are intended 
to excel beyond traditional classroom practices. For each of the 
441 models examined, tables were established using maximum 
likelihood probabilities to predict binary next problem correctness 
(time r + 1), given the partial credit score on the current question 
(time t). Findings suggest that a data driven approach to defining 
partial credit penalization is possible and that an optimal 
penalization range can be isolated using model accuracy. Further, 
findings suggest that within the optimal range, lower penalizations 
do not differ significantly from higher penalizations, allowing 
leeway for content developers and teachers to enhance student 
motivation through reduced penalization. 

2.3 Goals & Insight Sought 

As I delve into my dissertation I expect my work to grow and 
meld into a unified construct surrounding the enhancement of 
student motivation and learning within adaptive tutoring systems. 
It is clear that the facets discussed here will link the two 
underlying realms of my research (i.e., randomized controlled 
trials and data mining), but it is not yet clear how. Through 
continued investigation of feedback, student choice, and 
assessment methodologies, I hope to establish a unique line of 
research that remains broad and yet powerful. Advice on how to 
drive a broad topic dissertation is sought. Essentially, I hope to 
gain an external expert’s opinion on how to best merge the facets 
of my research. Advice on future endeavors within individual 
facets would also be appreciated. 

The immediate impact of my research is already evident through 
continued improvements to the ASSISTments platform. The work 
presented here has inspired content expansion as well as 
infrastructure changes to enhance future research design. Within 
the next three years I expect that my research will continue to 
refine ASSISTments while increasing intellectual merit in my 
field. The broader impact of my work will be measured in long- 
term achievements that affect systemic change in education and 
promote data driven practices and individualized learning via 
adaptive tutoring platforms. 
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